The Hebrew University of Jerusalem

We study non-zero-sum continuous-time stochastic games, also known as continuous-time Markov games, of fixed duration. We concentrate on Markovian strategies. We show by way of example that equilibria need not exist in Markovian strategies, but they always exist in Markovian publicsignal correlated strategies. To do so, we develop criteria for a strategy profile to be an equilibrium via differential inclusions, both directly and also by modeling continuous-time stochastic as differential games and using the Hamilton-Jacobi-Bellman equations. We also give an interpretation of equilibria in mixed strategies in continuous-time, and show that approximate equilibria always exist.